Imaging elements used in imaging devices such as digital cameras are configured such that a color filter made up of an RGB array for example is mounted thereon, and specific wavelength light is incident upon pixels.
Specifically, a color filter having a Bayer array for example is often used.
A Bayer-array captured image is what is known as a mosaic image in which only pixel values corresponding to any of the RGB colors are set to the pixels of an imaging element. A signal processing unit of a camera carries out demosaic processing and so forth in which various signal processing such as pixel value interpolation is carried out with respect to this mosaic image and all pixel values for RGB are set to the pixels, and a color image is generated and output.
Many studies have already been carried out with regard to signal processing for captured images for which a color filter according to this Bayer array is provided, and this signal processing can, to an extent, be said to be technically established. However, at present, sufficient studies have not yet been carried out with regard to signal processing for images having an array that is different from a Bayer array.
Signal processing in general cameras is often set to be executed with respect to Bayer array images, and if the pixel array of the imaging element is an array that is different from a Bayer array, it is possible to apply known signal processing by converting the pixel array of an image that is input from the imaging element into a Bayer array and inputting this to a camera signal processing unit.
Therefore, with regard to a captured image of a pixel array other than a Bayer array, it is preferable to execute processing for converting an input image from the imaging element into a Bayer array as preliminary processing prior to inputting to the camera signal processing unit. This kind of pixel array conversion processing is called “remosaic processing”. However, there are various arrays for imaging element pixel arrays, and there is no conventional technology that sufficiently discloses optimal remosaic processing with respect to these various pixel arrays.
It should be noted that configurations having an array that is different from a Bayer array are described in, for example, Patent Document 2 (JP 11-29880 A) and Patent Document 3 (JP 2000-69491 A).
In these Patent Documents 2 and 3, settings are implemented with which a plurality of same-color pixels, such as R pixels having 2×2 pixels, G pixels having 2×2 pixels, and B pixels having 2×2 pixels, are arranged in an imaging element (image sensor), the constituent pixels having 2×2 pixels of the same color are set to different exposure times, and photographing is executed. Patent Documents 2 and 3 describe configurations in which same-color pixel values of different exposure times captured by means of this kind of image sensor are synthesized, and wide dynamic range images are obtained.
However, these documents describe the generation of wide dynamic range images based on the synthesis of pixel values of a plurality of different exposure times, and a clear explanation with respect to remosaic processing is not disclosed.